High definition media storage structure and playback mechanism

ABSTRACT

An apparatus and method for storing and playing high definition content is disclosed. This invention provides a mechanism for storing and playing back high definition content on a medium such as DVD optical disc. One aspect of the invention is that elementary streams may be multiplexed and processed in a high definition media player instead of at authoring time. Another aspect of the invention is that it provides for extended real-time features such as inserting watermarks into the content stream, decrypting selected sections of the content stream, and performing trick playback display modes.

CROSS-REFERENCE TO RELATED APPLICATIONS.

This application is a continuation of application Ser. No. 09/540,557,filed Mar. 31, 2000, now U.S. Pat. No. 6,865,747, which claims thebenefit of Provisional Application No. 60/127,394, filed Apr. 1, 1999,the contents of which are incorporated by reference herein.

FIELD OF THE INVENTION

The present invention relates to high definition media storagestructures and playback mechanisms.

BACKGROUND OF THE INVENTION

Mechanisms for storage and processing of digital content on variousmedia have been defined for various digital content playback systems.Recently, the resolution of digital content has increased. This contentis now referred to as high definition digital content (HDDC). Currentstorage structures and playback mechanisms were not designedspecifically for HDDC. There is a need for new storage structures andplayback mechanisms for HDDC that introduce as little impact on currentstorage structures and play back mechanisms as possible. These newstorage structures and playback mechanisms will preferably supportmethods to prevent unauthorized access to the HDDC and to track anyunauthorized access to HDDC. It is also desired that these newstructures and playback mechanisms will support trick playback modes.The present invention broadly relates to and provides a solution tothese problems.

While the description which follows may sometimes be described in thecontext of audio/video/data as an example of content, the invention isnot so limited and may equally apply to any type of information orcontent data, including without limitation audio and/or video data orother type of data or executables.

The invention is described in terms of the current best mode. This bestmode is described as extensions of the DVD Specifications for Read-OnlyDisc (described in “DVD Specifications for Read-Only Disc”, Version 1.1,December 1997 by Toshiba Corporation) to support high resolution,encrypted and actively watermarked content. Media conforming to theseextensions are referred to in this document as HD-DVD. Playbackmechanisms which present the HDDC content to an ATSC/HDTV compatiblereceiver are also disclosed. These mechanism allow graphics, trickmodes, and watermarking to be extended to HDTV. One skilled in the artcan see that although the present invention is described in terms ofHD-DVD, the invention may be practiced on any digital storage mediaincluding hard disks, magnetic tape, and other optical discs.

The present application is directed to the same general technology asco-pending commonly assigned patent application Ser. No. PCT/US00/00079,entitled “Content Packet Distribution” naming Schumann et al. asinventors (the contents of which are incorporated by reference herein).This application is directed more to specific storage structures andplayback mechanisms including watermark insertion, trick modes, and ATSCstream generation.

In some commercial applications, where the content includes, forexample, valuable audio or video content, unauthorized access by thosewho obtain the content may tend to reduce the profit margin of thecontent provider(s), who typically provide the content, e.g. to variouslistener and/or viewers, for a fee. In particular, with the advent ofhigh definition video, this problem is even more serious because thedigital data is of sufficient resolution to be shown on a full sizetheater screen. This opens up a whole new area for content pirates tomarket their stolen property. If the unauthorized accesser is a contentpirate, he or she may pose a serious threat to a content provider byinducing others to pirate the content as well. More particularly, thepirate may generally sell pirated access to the content at a lower costthan the legitimate content provider because the pirate obtains accessto the content by using the legitimate provider's infrastructure andtherefore does not have to invest resources to produce and disseminatethe content. This becomes even a greater concern where the pirate maycopy and mass produce a relatively inexpensive component which allows alarge number of users to obtain access to the content withoutauthorization by the legitimate content provider. As a result, contentproviders have resorted to increasingly expensive and complex schemes toprevent unauthorized access to their information and content, i.e. toprevent pirating.

What is needed is a system and method for protecting valuable content; amethod and system which is robust, which may be tailored to the needs ofa particular content provider, and which overcomes the above noteddeficiencies.

SUMMARY AND ADVANTAGES OF THE INVENTION

One advantage of the invention is that it allows a disc to be authoredwhere the disc may be played by both conventional media players and highdefinition media players.

Another advantage of this invention is that elementary streams may bemultiplexed and processed in the high definition media player instead ofat authoring time.

Yet a further advantage of this invention is that it provides forextended real-time features such as inserting watermarks into thecontent stream, decrypting selected sections of the content stream, andperforming trick modes.

To achieve the foregoing and other advantages, in accordance with all ofthe invention as embodied and broadly described herein, an apparatus forplaying high definition content comprising a media player for receivingthe high definition content from a media source. The high definitioncontent is contained in data packets and the data packets are containedin sectors. A content processor processes the high definition contentinto transport packets and a transport packet modulator modulates thetransport packets. A controller manages the operations of the apparatus.

In yet a further aspect of the invention, the apparatus for playing highdefinition content further includes a watermark buffer for receivingwatermark data; a video buffer for receiving video data; an audio bufferfor receiving audio data; a watermark inserter for inserting watermarksinto the video data, determined by the video data and watermark data; acontent multiplexer; and a transport packet generator.

In yet a further aspect of the invention, preselected bocks of the datapackets are encrypted.

In yet a further aspect of the invention, the apparatus further includesa trick mode processor that can: create a slow motion effect byinserting empty predictive frames into a video elementary stream betweenpicture frames; create a pause effect by iteratively inserting into thevideo elementary stream a sequence comprising an Intra-coded pictureframe; and a multitude of predictive frames; create a fast forwardplayback effect by inserting forwardly sequenced Intra-coded pictureframes interspersed with empty predictive frames into the transportpacket stream; and create a rewind playback effect by inserting reversesequenced Intra-coded picture frames interspersed with empty predictiveframes into the transport packet stream.

In a further aspect of the invention, a method for playing highdefinition content comprising: receiving the high definition contentfrom a media source, the high definition content contained in datapackets and the data packets contained in sectors; processing the highdefinition content into transport packets; modulating the transportpackets; and outputting the modulated transport packets.

Additional objects, advantages and novel features of the invention willbe set forth in part in the description which follows, and in part willbecome apparent to those skilled in the art upon examination of thefollowing or may be learned by practice of the invention. The objectsand advantages of the invention may be realized and attained by means ofthe instrumentalities and combinations particularly pointed out in theappended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawing, which are incorporated in and form a part ofthe specification, illustrate an embodiment of the present inventionand, together with the description, serves to explain the principles ofthe invention.

FIG. 1 is a block diagram of a high definition content authoring system.

FIG. 2 is a block diagram of an embodiment of an aspect of the presentinvention used to playback high definition content.

FIG. 3A is a block diagram showing an example of a video data packingformat.

FIG. 3B is a block diagram showing an example of a video data packingformat header.

FIG. 3C is a block diagram showing another example of a video datapacking format header.

FIG. 4 is a diagram depicting timestamp calculations from a video bitstream.

FIG. 5 is a block diagram of an ATSC transport packet.

FIG. 6 is a block diagram showing how video access unit data may beencrypted as per an embodiment of the invention.

FIG. 7 is a block diagram showing alignment of encryped data in thetransport payload as per an embodiment of the invention.

FIG. 8 is a block diagram of watermark sectors as performed by somecurrent non-HD systems.

FIG. 9 is a block diagram of HD watermark sectors as performed by anexemplary aspect of the present invention.

FIG. 10A is a block diagram of an exemplary aspect of the presentinvention depicting watermark markers in a frame of video data.

FIG. 10B is a block diagram of an exemplary aspect of the presentinvention depicting a watermark marker structure.

FIG. 11 is a block diagram of an exemplary aspect of the presentinvention depicting a content processor.

FIG. 12 is a block diagram showing trick mode processing.

FIG. 13 is a block diagram showing how the slow motion playback trickmode can be obtained by inserting empty B pictures.

FIG. 14 is a block diagram showing data flow through an embodiment ofthe present invention.

FIG. 15 is a block diagram showing an embodiment of the presentinvention wherein watermarks are inserted into the content in the HDTV.

DETAILED DESCRIPTION OF THE INVENTION

The present invention provides for storing high definition content on aDVD or other storage media by extending the current specification of DVDread-only disc. The global disc layout may remain identical, preservingsoftware investments for DVD authoring tools & player firmware, buthigher video resolution and bit rates are allowed. HD-DVD players maynot need any MPEG-2 video or AC-3 audio decoders, but may use instead areal-time content processor 214 and a modulator 216. This invention alsoprovides for various “trick modes”, including fast forward, reverse,slow motion, and pause. Also provided for are mechanisms that may allowthe content to be encrypted and watermarked.

Encryption may be done on video, audio or other elementary streamsduring authoring, and may be based on blocks of consecutive bytes.Alignment methods ensures the mapping of encrypted blocks to the payloadof a transport packet, and some rules define the conditions under whicha block may or may not be encrypted, and where an encrypted block has tostart. The transmission of watermarks in encrypted format to the TVreceiver follows a buffering method and individual watermarks may begrouped in time stamped access units. Trick modes are also possible byslightly altering the content of video access unit headers and byinserting or suppressing MPEG-2 video frames. Finally, backwardcompatibility of, the new system is possible if the audio and videoformats of the classic DVD are supported by the ATSC standard (AC-3audio, MPEG-2 video). MPEG graphics may also be supported.

Reference will now be made in detail to the presently preferredembodiments of the invention, examples of which are illustrated in theaccompanying drawings.

A block diagram of a high definition content authoring system is shownin FIG. 1. Authoring systems are used to create the final image of thedigital content in a format compatible with the intended display system.Time stamped elementary input streams 100, including audio and videoinput streams formatted as MPEG-2 video and AC3 audio respectively, maybe multiplexed together by the authoring tool 102 as they would be witha classic DVD, including data for multiple angles and parental levels.The authoring tool 102 is in charge of creating system files for VideoManager and Video Title Sets, one or more MPEG-2 program streamincluding navigation packs (Video Objects), and storing them as a discor set of discs 104. In the presently illustrated embodiment, the discs104 may be UDF formatted.

FIG. 2 is a block diagram of an embodiment of an aspect of the presentinvention used to playback high definition content. The disc(s) 200containing authored content may be inserted in a HD-DVD enabled player210. The player has a media player 212 that process the DVDspecifications and extensions logic 220 that process the HD-DVDextensions. Together, the media player logic 212 and extensions logic220 interpret the contents of the disc(s) 200 to create a combination ofgraphic menus and audio/video streams.

An embodiment of the present invention may include a secure sessionmodule 218, so that secured communication may be established with thereceiver before content playback may be authorized.

To send data to the receiver, the program streams stored on the disc200, or generated by an MPEG graphic engine are converted to anATSC/HDTV transport stream by the content processor 214. The stream isthen modulated by a modulator 216 and sent to an HDTV receiver 224. Thepresent embodiment used 8-VSB modulation, however, any type ofmodulation capable of transporting the digital content may be used.

The receiver 224 demodulates the signal and reconstructs the transportstream packets, which are sent to their intended destinations which mayinclude a decryption encoder 226, audio and video decoders, or watermarklogic 228.

DVD Read Only vs. HD-DVD Specifications

The HD-DVD extensions may closely follow the DVD Specifications forRead-Only Disc, may enlarge the range of video formats allowed and mayallow higher data rates on the disc. In the embodiment of the presentinvention, wherein compatiblity with the ATSC/HDTV standard is desired,only those video and audio formats defined in the ATSC/HDTV standard maybe used when the HD-DVD contains high definition material intended forHDTV display. The term ‘HD mode’ refers to when the HD-DVD player playsa disc with features not found in the DVD specifications for Read-Onlydisc.

A detailed description of the new parameter boundaries and constraintsmay be defined, in particular in the interleaved units minimum jumpsizes that a HD-DVD player has to meet for multi-angle blocks. Areasthat that may be extended for HD-DVD includes: parts of the videoobjects such as the contents of VOB, the player reference model, thepresentation video, audio and sub-picture unit data; restrictions forseamless play; restrictions of SP_DCCMDs; relation between Informationin disc and player; display mode; position and allowed line number'srange of video and sub-picture; and karaoke mode in MPEG-2 audio.

In HD mode, the MPEG-2 video and audio configuration (i.e. bit rate,profile, level, AC3 for audio) may need to meet the ATSC/HDTV standardrequirements. Audio and video streams stored as an MPEG-2 program streamon the disc may be converted to an ATSC/HDTV compliant transport streamby the Content processor 214.

Audio & Video Demultiplexing

Extracting audio and video access units may be performed by extractingthe payload of each corresponding packet. A start code search may bedone to find each access unit's boundaries, unless pointers are added onthe disc as private data. As shown in FIG. 3A, each MPEG video accessunit (VAU) 300 may include a header 301 and slice data 302, whereheaders may comprise one of the two descriptions as illustrated in FIGS.3B and 3C.

FIG. 3B is a block diagram showing an example of a video data packingformat header 310 which may include a sequence header 311, sequenceextensions 312, GOP (group of picture) headers 313, picture headers 314and picture extensions 315.

FIG. 3C is a block diagram showing another example of a video datapacking format header 330 which may include a picture header 331 and apicture extensions 332.

An AC3 audio frame may start with a sync word 0xB77 and be encoded atconstant bit rate, which makes frame extraction quicker.

Timing Constraints

Timestamps include several components including a Tref, a Decoding TimeStamp (DTS), and a Presentation Time Stamp (PTS). The DTS represents thetime to decompress the frame. The PTS represents the the time to presentthe frame. Tref is a temporal reference number. Obtaining a time stampfor each MPEG video access unit (PTS and DTS) can be done during VAUextraction, by using the PTS and DTS fields found in PES packet headersof DVD sectors. Only I frames are required to have PTS and DTS. Timestamps for other pictures can be computed, but at the expense of extramemory to store data until the next reference frame (the DTS of areference frame should be equal to the PTS of the previous referenceframe). Time stamps may also be inserted at DVD authoring time to reducememory requirements in the player.

An example of DTS and PTS computation from a video bit stream, when onlythe first PTS and DTS are known is illustrated in FIG. 4. Timestamps400, 401, 402, 403 and 404 represent a sequence of timestamps. Timestamp400 corresponds to an I frame. Timestamp 403 corresponds to a P framewhich is derived from an I frame. Timestamps 401, 402 and 404 correspondto B frames which may be derived by either P frames, I frames, or both.

AC3 audio streams may be encoded at constant bit rate, in which case thePTS can be computed by linear extrapolation.

Multiplexing Streams

A novel aspect of the present invention is that the transportmultiplexing module may run independently in the player. Previous artusually performs this task in the authoring tool. In the presentlyillustrated example, there are three inputs: a video stream, an audiostream, and private data streams, each access unit being sent with acorresponding set of time stamps. In movie play mode, the streams aremultiplexed according the MPEG-2 specifications, the private data streamfollowing a buffer model similar to the audio stream. The timestampedprivate stream may be used to assist in watermarking the content atruntime using a corresponding watermark access unit.

In MPEG graphics mode, the background is sent as an I frame and pictureelements are added using P frames. Techniques similar to trick mode playmay be used to build a valid, time stamped, MPEG-2 elementary streamthat may be sent to the transport multiplexing module.

Any DVD authored according to the DVD Specifications for Read Only Discwith video and audio format supported by the ATSC (MPEG-2 video and AC3audio) may offer a valid input for the transport multiplexer 214 and maybe sent to an ATSC compliant HDTV through an 8-VSB interface 222.

Storage, Format and Procedures to Handle Encrypted Content

Another novel feature of the present invention is that it allows contentto be independently encrypted on a block by block basis. The DVD storageformat is based on 2048 bytes per logical sector. This format is onlyused for storage, and the transmission of data may be done with ATSCtransport packets of 188 bytes:

FIG. 5 is a block diagram of a 188 byte ATSC transport packet 500. TheTransport packet 500 may include a 4 byte header 501, an adaption field502, and a payload 503. When a packet has no adaptation field 502, thepayload 503 may have a size of 184 bytes. When an adaptation field 502is present, to carry a PCR or padding bytes for example, the number ofbytes of the payload 503 may be reduced accordingly.

FIG. 6 is a block diagram showing how video access unit data may beencrypted as per an embodiment of the invention. To allow a real-timeconversion from a time stamped MPEG-2 video stream and a time stampedAC3 audio stream to a valid transport stream, some fields in the headers601 may have to be read and/or modified. For this reason, they may notbe encrypted. In a video access unit the encryption may start on thefirst 184 byte block 606 completely contained in the slice data area602, continue through blocks 607, 608, 609, 610, 611, 612 and stop onthe last 184 byte block 613 completely contained in the slice data area.Not all of these blocks have to be encrypted, but no other blocks may beencrypted in the video stream. Therefore, blocks 603, 604, 605 and 614are not encrypted.

Audio streams can be encrypted in a less restrictive manner, since thesize of an access unit can be predicted. For example, only 1 out of 10audio access units can be left unencrypted.

A major problem with encryption of elementary streams is to avoid anymisalignment between elementary stream encrypted data and transportstreams packet decryption. FIG. 7 is a block diagram showing alignmentof encrypted data in the transport payload as per an embodiment of theinvention. Transport packet 700 contains a 4-byte header 701, alignmentpadding bytes 702, and payload bytes 703. The second transport packet710, includes a 4 byte header 711 and 184 bytes of encrypted payloaddata 712. A solution to this problem, if it occurs, is to insert paddingbytes 702 in the last packet 700 preceding a group of encrypted packet710 to ensure the correct alignment of the 184 bytes of the transportpacket payload 712 (a transport packet cannot contain both encrypted andunencrypted data). If the data is encrypted as previously described,then only one padding operation is required, in the last packetpreceding the blocks of encrypted slice data. When an adaptation fieldmust be sent during the transmission of encrypted packets, to transmitPCR for example, then an extra transport packet may be inserted with anadaptation field but no payload at all, in order to preserve theencryption alignment.

A method to signal in each frame which 184 bytes block is encrypted, andwhich one is not is now described. A header made of a few bytes in eachDVD sector is used. One byte indicates the number of bytes in thepayload before the beginning of the first 184 byte block. Then 11 groupsof 2 bits may be used to store the MPEG-2 transport scrambling controlfield. One bit indicates if the sector contains any encrypted data. Atotal of 31 bits may be required. Those bits may be stored over anunused DVD sector packet header field, like SCR, when the VOBS hasencrypted content. Another option may be to simply encrypt all data, andset a flag in a global header.

In summary, video elementary stream may be partitioned in blocks of 184consecutive bytes, and each of these blocks which only contain slicedata and only slice data can be encrypted. To restore the alignment ofthese blocks with the payload of a transport packet, padding bytes maybe used in the adaptation field of the transport packet preceding anencrypted transport packet. To preserve the alignment of the payloadwhen an adaptation field is required, an extra packet with no payloadmay be inserted.

Allowing the occasional insertion of padding bytes and packets withoutpayload, the bit rate of the video elementary stream may be carefullyadjusted to avoid any video buffer overflow. This constraint may becombined with the bandwidth requirements of watermark information.

Watermarking Support

An example of current watermark technology is illustrated in FIG. 8.Watermark sectors 800 are performed on this non-HD systems by insertingprivate sectors 801 at authoring time to store watermark information(mainly replacement data and location for each watermark). The locationof a watermark is identified by 3 parameters: A physical sector number802, an offset in the sector 803, and the length in bytes 804. When asector 831, 832, and 833 is received from data on the disc 830, thesector number is compared with those in the watermark table and if areplacement is required, replacement data 805 is written into the sector832 at the location indicated by the offset.

Refering to FIGS. 9, 10A, and 10B, we will now discuss the aspect of thepresent invention that implements content watermarking. FIG. 9 is ablock diagram of HD running sectors as performed by an exemplary aspectof the present invention where watermark technology may be preserved byonly changing location information of the watermarks. FIG. 10B is ablock diagram of an exemplary aspect of the present invention depictinga watermark marker structure.

In the HD-DVD context, the receiver is in charge of inserting watermarksand has no knowledge of DVD physical sectors. A solution to this problemis to assign an identifier to each watermark and insert the identifierin the video where the watermark must be inserted as illustrated in FIG.10. In FIG. 10A, markers 1002, 1004, and 1006 are inserted in the videocontent 1000 where a watermark is intended to be written. The markers1050 includes a start code 1052 and a watermark ID 1054. One skilled inthe art will recognize that many different marking schemes may be usedto indicate locations for watermark insertions.

As presently illustrated, the current embodiment uses an 8 bytewatermark that is overwritten. The ID 1052 may be a 4 byte longwatermark start code such as 0x000001BA. The sequence number 1054 may bea 4 byte unique watermark identifier, WMID 920. The watermark sectornumber 802 and offset 803 used in the prior art are replaced by the WMID920 in the watermark sector. The original 8 bytes of data may be savedin the watermark sector and the PTS of the picture to which thewatermark applies allows the transport stream multiplexer 214 to sendthe watermark data in real-time. For example, the WMID 920 may be anincrementing counter starting at 0x0200 to avoid generating start codes.

An alternative method would to use the WMID/offset 920 as an offset intothe frame. This method would not require any markers in the video data.

As described above, the watermarks are stored in HD watermark sectors900. A group of watermarks with the same PTS 922 may be referred as awatermark access unit and may be stored in the same physical sector.This access unit may follow a watermark buffering model, which may bedescribed with a leak rate and buffer size that may be defined dependingon the bandwidth allowed for watermarks (This buffering model isdescribed the MPEG-2 standard). The transport multiplexer 214 may ensurethat each access unit arrives in time in the watermark buffer.

When a picture is received and watermarks have to be inserted, thepicture may be scanned for the watermark start codes which are followedby the WMID 920. The watermark buffer 910 has the corresponding WMID 920information to either restore the original 8 bytes 924 (start code andWMID) or to insert the replacement data 928. The size of the replacementdata 926 may be stored as part of the watermark buffer 910. If thecorresponding WMID 920 is not in the buffer, then a pirate attack isvery likely to have occurred. The TV may decide to wait a few secondsand turn the screen dark, refusing content playback.

If the start code search method is too demanding in CPU resources in theTV, an offset from the first byte of the slice data could indicate thelocation of each watermark.

Encryption of watermarks may be done on a watermark access unit level.Watermarks belonging to the same frame (i.e. watermarks having the samePTS) may be grouped together in a more efficient manner to allowencryption: a watermark access unit header followed by watermark data.The header could be composed of the DTS, number of watermarks in theaccess unit, size in bytes, and would not be encrypted. The rest of thedata could be encrypted and aligned by the transport multiplexer 214with the same method that for video access units. Not encrypting theheader should not compromise the security of the system since the WMIDfound in the picture at watermark insertion time must match thewatermark data, and watermarks attacks can be detected.

FIG. 11 is a block diagram of an exemplary aspect of the presentinvention depicting a content processor 1110 that is configured to inputelementary streams, insert watermarks, perform trick mode displayfunctions, multiplex audio and video content, and formats the resultantdata into a valid output transport stream 1134 such as ATSC for output.The elementary streams include watermarking packets 1100, video packets1102, and audio packets 1104. The watermarking packets are input into awatermarking buffer 1120. A watermark inserter 1126 inputs watermarksectors from the watermark buffer 1120 and inserts watermarks into thevideo data stored in a first video buffer 1122 for output into a secondvideo buffer 1128 using watermarking techniques that were discussedpreviously. The video packets 1102 are input into a first video buffer1122. The data is then transferred into a second video buffer 1128 wherethe video data is combined with watermarks. Next, the data stored in thesecond video buffer 1128 may be input to a trick mode processor 1130where output display trick modes may be performed on the video streams.The audio packets 1104 are input into an audio buffer 1124. Dataprocessed by the trick mode processor 1130 and the audio buffer 1124 areboth input into a content multiplexer 1132 which combines the data intoa combined data stream. The combined data stream is then input into atransport packet generator 1134, which formats the data into a transportpacket stream 1140 such as ATSC. One skilled in the art will recognizethat a content processor could be built to handle other types of datainstead of or in combination with the watermark, video and audio datatypes discussed here.

FIG. 15 shows a block diagram of another embodiment of the presentinvention demonstrating how video watermark insertion may occur in anHDTV 1520. As illustrated, the content is contained on a media 1500. Thecontent is read and processed by an HD player 1510 which produces atransport stream such as ATSC and modulated using a modulation schemesuch as 8-VSB, containing processed content for display. The processedcontent is input into the HDTV, where it may be decrypted anddemultiplexed by a decrypter/demultiplexer device 1530. Next the data isstored into buffers. In this example, the timestamped watermarkelementary stream is buffered in a timestamped watermark elementarystream buffer 1532. The timestamped video elementary stream is bufferedin a timestamped video elementary stream buffer 1534. Both the bufferedwatermark and video data are input into a watermark inserter 1538 wherewatermarks are inserted into the video stream producing a watermarkedvideo stream 1540 such as MPEG video. The watermarks are inserted whenthe timestamps (DTS/PTS) match the current Video picture DTS/PTS timestamps The watermarked video stream 1540 is then displayed as a decodedimage 1542.

One skilled in the art will appreciate that the concept of watermarkingas presented may equally be applied to other types of data streamsbesides video, such as audio, executable or process data. Executabledata may include programs intended for execution on a target device suchas a smart HDTV. Process data may include data or files that communicateinformation such as HTML or XML to a target device.

Trick Modes

Trick modes modify the video stream to produce output display effectssuch as pause, slow motion, fast forward and reverse. Traditionally,trick modes are generated directly by decompression chips. The presentinvention may generate trick modes altering the video stream before itis decoded. FIG. 12 shows a video stream 1200 being input to a trickmode processor 1210. The output of the trick mode processor 1210 is amodified video stream 1220 that may now be multiplexed with othercontent streams before being converted into a stream of transportpackets. The video frames are typically MPEG frames. MPEG frames includeP frames, B frames and I frames. I frames, are video frames known asIntra-coded pictures (I-pictures). I frames are coded in such a way thatthey can be decoded without knowing anything about other pictures in avideo sequence. P frames, are video frames known as predictive codedpictures (P-pictures). P frames are decoded using information fromanother frame that was displayed earlier. B frames, are video framesknown as bidirectionally predicted pictures (B-pictures). B frames arebi-directionally decoded using information from other frames. The otherframes may occur before or after the B frame. P frames and B frames areoften referred to as predictive frames. Trick modes may be achieved byextracting MPEG-2 video elementary frames using search algorithms. Theframes may be converted to a valid MPEG-2 video elementary stream byadjusting headers, like the temporal reference fields of picture headersand by inserting empty P frames or empty B frames. An empty frame hasnull motion vectors, no residual data coded (coded block pattern is 0)and has the property of repeating the content of one of the referenceframes. These techniques, along with a time stamp correction providesthe possibility to generate a valid MPEG-2 elementary video stream witha valid number of frames per second (29.97 for NTSC for example). Theimpact on the content processor 214 is that it must continuously outputthe data stream. A stack of queued transport packets transferred inhardware by DMA may reduce the amount of CPU required.

Each picture header and PES (program elementary stream) header may bechanged to reflect the insertion or deletion of pictures in theelementary video stream. Because of the interdependency of I, P and Bframes, some rules may need to be followed including: (1) any B framemay be suppressed or inserted; (2) a P frame may be suppressed only ifall other frames dependent upon the suppressed P frame are alsosuppressed.

FIG. 13 shows a sequence of frames where the first frame 1300 is a firstoriginal picture. Empty B frame(s) 1310 may be inserted into the videostream to create a slow motion or pause effect. Then a second originalpicture 1320 is input to the video stream. Fast forward and rewindplayback may be obtained by playing back I frames and inserting empty Bframes to adjust playback speed and control the bitrate.

Although the trick modes are described here in terms of MPEG-2 frames,one skilled in the art will recognize that the present invention may bepracticed on other types of video that utilize predictive video frames.

Data Flow

FIG. 14 is a block diagram that shows the data flow through anembodiment of the present invention during playback of high definitioncontent with real-time conversion to a packet transport stream. The highdefinition digital content is authored and stored on storage media 1400.An HD player 1410 may then read the content from the media 1400. Thecontent may be stored as audio, video, and data sectors 1420. An exampleof a storage media may be a classic DVD disc, extended with HighDefinition Video formats and an example of a data sector type may be DVDsector. A media player 1430 may include a media reader and media readerlogic. Data may be extracted from the sectors 1430 and demultiplexedinto elementary streams including an elementary video stream 1440, anelementary watermark stream 1442, and an elementary audio stream 1444.Timestamps may be included within the elementary data streams. Thesestreams may be input to a content processor 1450 where they may beprocessed. Processes may include insertion of watermarks, processingtrick output display modes, multiplexing content streams, and transportpacket generation. The output of the content processor 1450 may betransport packets such as ATSC transport packets. Content multiplexingmay need to follow packet alignment methods to ensure valid decryptedelementary streams when the streams are encrypted. A modulator 1460 maymodulate the output packets for transport to an HDTV 1470. The HDTV mayalso perform functions on the content including demultiplexing theelementary streams, decoding the content, decrypting the content,watermarking the content and displaying the content.

The present invention provides extensions of media formats including DVDto high resolution video, while maintaining most of the currentarchitectures. An added benefit of this invention is backwardcompatibility, although backward compatibility may be limited to someaudio and video format. These HD-DVD extensions provide for encryptingcontent, watermarking content, and trick playback display modes.

Although the present invention has been fully described by way ofexamples with reference to the accompanying drawings, it is to be notedthat various changes and modifications will be apparent to those skilledin the art. For example, it will be apparent to those of skill in theart that the content may be provided from any type of source device forprocessing and playback on other devices according to principles of thepresent invention. Therefore, unless such changes and modificationsdepart from the scope of the present invention, they should be construedas being included therein.

1. An apparatus for playing high definition content comprising: (a) ahigh definition media player for receiving multiplexed high definitioncontent from a media source, said multiplexed high definition contentand timestamps contained in data packets; (b) a content processor forprocessing said multiplexed high definition content into transportpackets; and (c) an HDTV comprising: (1) a decrypter; (2) ademultiplexer; (3) a watermark buffer for receiving watermark data; (4)a video buffer for receiving video data; and (5) a watermark inserterfor inserting watermarks into the video data, determined by the videodata and watermark data.
 2. The apparatus according to claim 1, whereinsaid data packets are contained in sectors.
 3. The apparatus accordingto claim 1, further including a transport packet modulator formodulating said transport packets.
 4. The apparatus according to claim1, wherein said data packets comprise at least one of: (a) watermarkdata; (b) video data; (c) audio data; (d) executable data; and (e)process data.
 5. The apparatus according to claim 1, wherein preselectedblocks of said data packets are encrypted.
 6. The apparatus according toclaim 1, wherein said media source is an optical disc.
 7. The apparatusaccording to claim 2, wherein said sectors are DVD sectors, and saidcontent processor generates an ATSC transport system.
 8. The apparatusaccording to claim 4, wherein said watermark data includes at least oneof: (a) a watermark identifier; (b) an offset: (c) a presentation timestamp; (d) original data; and (e) size data.
 9. The apparatus accordingto claim 4, wherein said video data includes at least one of thefollowing: (a) a start code; and (b) a watermark identifier.
 10. Theapparatus according to claim 1, wherein said content processor furtherincludes a trick mode generator.
 11. The apparatus according to claim10, wherein said trick mode generator can create a slow motion effect byinserting empty predictive frames into a video elementary stream betweenpicture frames, wherein the rate of said slow motion effect isdetermined by the quality of predictive frames instead.
 12. Theapparatus according to claim 10, wherein said trick mode generator cancreate a pause effect by inserting into a video elementary stream amultitude of predictive frames.
 13. The apparatus according to claim 10,wherein said trick mode processor can create a fast forward playbackeffect by inserting forwardly sequenced Intra-coded picture framesinterspersed with empty predictive frames into a transport packetstream, wherein the rate of said fast forward motion effect isdetermined by the quantity of Intra-coded picture frames and predictiveframes.
 14. The apparatus according to claim 10, wherein said trick modeprocessor can create a rewind playback effect by inserting reversesequenced Intra-coded picture frames interspersed with empty predictiveframes into a transport packet stream, wherein the rate of said rewindplayback effect is determined by the quantity of Intra-coded pictureframes and predictive frames.
 15. The apparatus according to claim 10,wherein said trick mode processor can create a playback effect byinserting Intra-coded picture frames interspersed with frames into atransport packet stream.
 16. A method for playing multiplexed highdefinition content comprising: (a) receiving multiplexed high definitioncontent from a media source, said multiplexed high definition contentand timestamps contained in data packets; (b) processing saidmultiplexed high definition content into transport packets; and (c)outputting said transport packets to an HDTV, said HDTV comprising: (1)a decrypter; (2) a demultiplexer; (3) a watermark buffer for receivingwatermark data; (4) a video buffer for receiving video data; and (5) awatermark inserter for inserting watermarks into the video data,determined by the video data and watermark data.
 17. The methodaccording to claim 16, wherein said data packets are contained insectors.
 18. The method according to claim 16, further including thesteps of: (a) modulating said transport packets; and (b) outputting saidmodulated transport packets.
 19. The method according to claim 16,wherein said step of receiving said multiplexed high definition contentfrom a media sources comprises reading content from an optical disc. 20.The method according to claim 19, wherein said optical disc is a DVD andsaid step of reading content from an optical disc further comprisesreading DVD sectors from said optical disc.
 21. The method according toclaim 16, wherein said step of processing said multiplexed highdefinition content into transport packets further includes generating aslow motion effect by inserting empty predictive frames into a videoelementary stream between picture frames, wherein the rate of said slowmotion effect is determined by the quantity of predictive framesinserted.
 22. The method according to claim 16, wherein said step ofprocessing said multiplexed high definition content into transportpackets further includes generating a fast forward playback effect byinserting forwardly sequenced Intra-coded picture frames interspersedwith empty predictive frames into a transport packet stream, wherein therate of fast forward motion effect is determined by the quantity ofIntra-coded picture frames and predictive frames.
 23. The methodaccording to claim 16, wherein said step of processing said multiplexedhigh definition content into transport packets further includesgenerating a rewind playback effect by inserting reverse sequencedIntra-coded picture frames interspersed with empty predictive framesinto a transport packet stream, wherein the rate of said fast forwardmotion effect is determined by the quantity of Intra-coded pictureframes and predictive frames.
 24. The method according to claim 16,wherein said step of processing said multiplexed high definition contentinto transport packets further includes creating a playback effect byinserting Intra-coded picture frames interspersed with frames into atransport packet stream.